EVALITA Evaluation of NLP and Speech Tools for Italian Proceedings of the Final Workshop

Editor of the proceedings of EVALITA 2016

Raising interest and collecting suggestions on the EVALITA evaluation campaign

This paper describes the design and reports the results of two questionnaires. The first of these questionnaires was created to collect information about the interest of industrial companies in the field of Italian text/speech analytics towards the evaluation campaign EVALITA; the second to gather comments and suggestions for the future of the evaluation and of its final workshop from the participants and the organizers of the campaign on the last two editions (2011 and 2014). Novelties introduced in the organization of EVALITA 2016 on the basis of the questionnaires results are also reported

Raising Interest and Collecting Suggestions on the EVALITA Evaluation Campaign

This paper describes the design and reports the results of two questionnaires. The first of these questionnaires was created to collect information about the interest of industrial companies in the field of Italian text/speech analytics towards the evaluation campaign EVALITA; the second to gather comments and suggestions for the future of the evaluation and of its final workshop from the participants and the organizers of the campaign on the last two editions (2011 and 2014). Novelties introduced in the organization of EVALITA 2016 on the basis of the questionnaires results are also reported

Pitch and functional characterization of hesitation phenomena in Italian discourse

Schettino L, Betz S, Cutugno F, Wagner P. Pitch and functional characterization of hesitation phenomena in Italian discourse. Presented at the Phonetics and Phonology in Europe, Barcelona, Spain

EVALITA: evaluation of NLP and Speech Tools for Italian

Comunicació presentada a: Final Workshop, celebrat a Naples (Florida) el 7 de desembre de 2016.EVALITA is the evaluation campaign of Natural Language Processing and Speech Tools for the Italian language: since 2007 shared tasks have been proposed covering the analysis of both written and spoken language with the aim of enhancing the development and dissemination of resources and technologies for Italian. EVALITA is an initiative of the Italian Association for Computational Linguistics (AILC, http://www.ai-lc.it/) and it is supported by the NLP Special Interest Group of the Italian Association for Artificial Intelligence (AI*IA, http://www.aixia.it/) and by the Italian Association of Speech Science (AISV, http://www.aisv.it/). In this volume, we collect the reports of the tasks’ organisers and of the participants to all of the EVALITA 2016’s tasks, which are the following: ArtiPhone - Articulatory Phone Recognition; FactA - Event Factuality Annotation; NEEL-IT - Named Entity rEcognition and Linking in Italian Tweets; PoSTWITA - POS tagging for Italian Social Media Texts; QA4FAQ - Question Answering for Frequently Asked Questions; SENTIPOLC - SENTIment POLarity Classification. Notice that the volume does not include reports related to the IBM Watson Services Challenge organised by IBM Italy, but information can be found at http://www.evalita.it/2016/tasks/ibm-challenge. Before the task and participant reports, we also include an overview to the campaign that describes the tasks in more detail, provides figures on the participants, and, especially, highlights the innovations introduced at this year’s edition. An additional report presents a reflection on the outcome of two questionnaires filled by past participants and organisers of EVALITA, and of the panel “Raising Interest and Collecting Suggestions on the EVALITA Evaluation Campaign” held at CLIC-it 2015

EVALITA 2016: Overview of the 5th evaluation campaign of natural language processing and speech tools for Italian

EVALITA1 is the evaluation campaign of Natural Language Processing and Speech Tools for the Italian language. The aim of the campaign is to improve and support the development and dissemination of resources and technologies for Italian. Indeed, many shared tasks, covering the analysis of both written and spoken language at various levels of processing, have been proposed within EVALITA since its first edition in 2007. EVALITA is an initiative of the Italian Association for Computational Linguistics2 (AILC) and it is endorsed by the Italian Association of Speech Science3 (AISV) and by the NLP Special Interest Group of the Italian Association for Artificial Intelligence4 (AI∗IA). Following the success of the four previous editions, we organised EVALITA 2016 around a set of six shared tasks and an application challenge. In EVALITA 2016 several novelties were introduced on the basis of the outcome of two questionnaires and of the fruitful discussion that took place during the panel "Raising Interest and Collecting Suggestions on the EVALITA Evaluation Campaign" held in the context of the second Italian Computational Linguistics Conference (CLiC-it 2015) (Sprugnoli et al., 2016). Examples of these novelties are a greater involvement of industrial companies in the organisation of tasks, the introduction of a task and a challenge that are strongly application-oriented, and the creation of cross-task shared data. Also, a strong focus has been placed on using social media data, so as to promote the investigation into the portability and adaptation of existing tools, up to now mostly developed for the newswire domain

EVALITA. Evaluation of NLP and Speech Tools for Italian : Proceedings of the Final Workshop 7 December 2016, Naples

EVALITA is the evaluation campaign of Natural Language Processing and Speech Tools for the Italian language: since 2007 shared tasks have been proposed covering the analysis of both written and spoken language with the aim of enhancing the development and dissemination of resources and technologies for Italian. EVALITA is an initiative of the Italian Association for Computational Linguistics (AILC, http://www.ai-lc.it/) and it is supported by the NLP Special Interest Group of the Italian Association for Artificial Intelligence (AI*IA, http://www.aixia.it/) and by the Italian Association of Speech Science (AISV, http://www.aisv.it/). In this volume, we collect the reports of the tasks’ organisers and of the participants to all of the EVALITA 2016’s tasks, which are the following: ArtiPhone - Articulatory Phone Recognition; FactA - Event Factuality Annotation; NEEL-IT - Named Entity rEcognition and Linking in Italian Tweets; PoSTWITA - POS tagging for Italian Social Media Texts; QA4FAQ - Question Answering for Frequently Asked Questions; SENTIPOLC - SENTIment POLarity Classification. Notice that the volume does not include reports related to the IBM Watson Services Challenge organised by IBM Italy, but information can be found at http://www.evalita.it/2016/tasks/ibm-challenge. Before the task and participant reports, we also include an overview to the campaign that describes the tasks in more detail, provides figures on the participants, and, especially, highlights the innovations introduced at this year’s edition. An additional report presents a reflection on the outcome of two questionnaires filled by past participants and organisers of EVALITA, and of the panel “Raising Interest and Collecting Suggestions on the EVALITA Evaluation Campaign” held at CLIC-it 2015